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A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) OR THIRTY (30) DAYS, 
WHICHEVER IS LONGER, FROM THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1 .704(b). 

Status 

1 )KI Responsive to communication(s) filed on 14 February 2008 . 
2a )^ This action is FINAL. 2b)D This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) |EI Claim(s) 22-25,27-32,34 and 35 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) |EI Claim(s) 22-25,27-32,34 and 35 is/are rejected. 

7) 0 Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) Q The specification is objected to by the Examiner. 

10) ^ The drawing(s) filed on 15 September 2003 is/are: a)^ accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 
Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 

1 1) D The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 119 

12) D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or (f). 
a)D All b)D Some * c)D None of: 

1 .□ Certified copies of the priority documents have been received. 

20 Certified copies of the priority documents have been received in Application No. . 

3.Q Copies of the certified copies of the priority documents have been received in this National Stage 
application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 
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DETAILED ACTION 



Claim Rejections - 35 USC § 103 

1 . The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

2. Claims 22-25, 27, 29-32, and 34 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Ezzat et al. (NPL document, "Visual Speech Synthesis by Morphing 
Visemes", herein referred to as "Ezzat") in view of Jiang et al. (NPL document, "Visual Speech 
Analysis with Application to Mandarin Speech Training", herein referred to as "Jiang") in view 
of Cox et al. (NPL Doc, "Speech and language processing for next-millennium communications 
services"). 

As per claims 22, 23, and 30, Ezzat teaches the claimed "selecting" step on top of 1 st 

column on pg. 5 1 and states: 

"there are many intermediate frames that lie between the chosen viseme images . . . 
Consequently, we compute a series of consecutive optical flowvectors between each 
intermediate image and its successor, and concatenate them all into one large flow vector 
that defines the global transformation between the chosen visemes". (emphasis added) 

And states in the abstract: 

we are able to synchronize the visual speech stream with the audio speech stream, and 
hence give the impression of a photorealistic talking face. 

(emphasis added) 



Application/Control Number: 10/662,550 
Art Unit: 2628 



Page 3 



Here, the visemes represent a generic facial image that can be use to describe a particular 
sound and the flowvectors which contain visual and sound features are used in conjunction with 
the visemes. 

Ezzat does not explicitly teach the claimed "obtaining" step. Jiang teaches the claimed 

"obtaining" step by stating in the abstract: 

At each frame, region of interest is identified and 
key information is extracted. The preprocessed acoustic 
and visual information are then fed into a modular TDNN 
and combined for visual speech analysis, (emphasis added) 

states on (pg. 1 14, 4.2 Acoustic and Visual Input Representation, 1st paragraph): 

For acoustic data representation, we have followed 
the well-established approach to apply FFT on the Hamming 
windowed speech data to get 16 Melscale Fourier coefficients as 
input to the Acoustic input Layer. For visual data representation, 
we have performed the lip-tracking and feature points extraction 
task by applying our 2D multi-state lip shape model. Then we 
use both the color profile of the feature points on external and 
internal boundaries and position and movement of lip boundaries 
for feature extraction using principle component analysis (PCA). 
The extracted feature vectors are then fed to the Visual Input 
Layer, (emphasis added) 

Here, the Jiang teaches feature vectors (target feature vector) and teaches of visual data 
(visual features) and acoustic information (non- visual information). It would have been obvious 
to one of ordinary skill in the art at the time of invention to combine Ezzat with Jiang. Jiang 
teaches one advantage to obtaining feature vectors in order to help children improve their speech 
pronunciation (see section 5, pgs. 114-115, 1 st paragraph) by providing audio-visual feedback. 



Ezzat does not explicitly teach the claimed "unit selection process". 
Cox teaches the claimed: 
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Unit selection process in which a longest possible candidate image sample is selected (in figure 
2 and bottom of 2 nd col on page 1318, "There are two good reasons why the method of unit 
selection synthesis is capable of producing customer-quality or even natural-quality speech 
synthesis. First, on-line selection of speech segments allows for longer units (whole words, 
potentially even whole sentences) to be used in the synthesis if they are found in the inventory . ". 
In addition, the references teaches of image samples, towards the middle of the 1 st col on page 
1319, "synthesized using photo-realistic two-dimensional image technologies (sample-based 
VTTS) " where the unit selection process is used with these samples because there is a 
correspondence between the audio and visual TTS). 

It would have been obvious to one of ordinary skill in the art at the time of invention to combine 
Ezzat, Jiang, and Cox. Cox teaches one advantage of the combination (bottom of 2nd col on 
page 1318, "There are two good reasons why the method of unit-selection synthesis is capable of 
producing customer-quality or even natural-quality speech synthesis) where this can improve the 
output of Ezzat. 

As per claims 24-25, and 31-32, Ezzat teaches the claimed "selecting . . . using a 

comparison of a combination of visual features and non-visual features with the target feature 

vector" by stating on pg. 47, 2 nd col, 2 nd paragraph: 

For any input text, we determine the appropriate sequence of viseme morphs to make, 
as well as the rate of the transformations by utilizing the output of the natural language 
processing unit (emphasis added) 
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In order to determine the appropriate sequence, the system would have to perform a comparison 
of visual and non-visual features with a given target vector in order to produce the output as 
stated. Further, this construction process of an appropriate sequence of viseme morphs would 
require selecting candidate image samples where these samples could be used to transition 
between through transformation. 

Ezzat teaches the claimed compiling by teaching of concatenation (see quote from top of 
1 st column on pg. 51 above). 

As per claim 27 and 34, Ezzat teaches the claimed first database by teaching of recording 
and collecting one image per English phoneme (bottom of 1 st column on pg. 47 under "Corpus 
and Viseme Acquisition", also see figure 2). 

Ezzat teaches the claimed second and third database by teaching of "Flow database" (pg. 
54, 2 nd column), which contain optical flow vectors which specify transition data between 
visemes (includes visual data and includes storing non-visual data i.e. sound transitions). 

As per claim 29, Ezzat teaches the claimed first database in figure 2, the claimed second 
database and the claimed third database on pg. 54, 2 nd column under "Flow database" where this 
database is formed to specify visual and non-visual data between animation transitions (frames). 

3. Claims 28 and 35 are rejected under 35 U.S.C. 103(a) as being unpatentable over Ezzat in 
view of Jiang in further view of Cox in further of view of Brand (NPL Document, "Voice 
Puppetry", herein referred to as "Brand"). 

As per claims 28 and 35, Ezzat does not teach the claimed limitations. 
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Brand teaches the claimed "selecting ... a number of candidates" and the claimed 

"Viterbi search" by stating on the bottom half of the 1 st col on pg. 25: 

The Viterbi sequence, while most likely, may only represent a small fraction of the total 
probability mass — there may be thousands of slightly different state sequences that 
are nearly as likely. If this were to happen in the voice puppet, V would be a very poor 
representation of the relevant information 
in the audio, and the animation quality would suffer greatly. 

. . . These problems are virtually banished with entropically estimated models because 
entropy minimization concentrates the probability mass on the optimal Viterbi 
sequence, (emphasis added) 

Brand teaches the claimed concatenation cost by stating on pg. 26, very bottom of 1 st col 

and very top of 2 nd col: 

We quantified this with a squared error measure of divergence between groundtruth (x) 
and reconstructed (y) facial motion vectors, weighted to penalize motions in the wrong 
direction, (emphasis added) 



It would have been obvious to one of ordinary skill in the art at the time of invention to combine 
Brand with the combinable system of Ezzat, Jiang, and Cox. Brand teaches the advantage of 
using an optimal Viterbi sequence with a large number of state sequences (candidates) to reduce 
the size to the most optimal ones in order to remove poor animation quality (1 st col on pg. 25 see 
quote above). 



Response to Arguments 

1 . Applicant's arguments filed 2/14/2008 have been fully considered but they are not 
persuasive. 
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Applicant argues that Cox et al. is not prior art because the reference falls under the exclusion of 
35 U.S.C. 103(c) (page 2 in filed response). 

The examiner respectfully maintains that the rejections are proper because the Cox et al. 
reference does not fall under the exclusion of 35 U.S.C. 103(c) when one considers looking at 
the issues in this case. The applicant's interpretation of the 103(c) is not completely correct. 
Cox et al. qualifies as prior art under 102(a) as a printed publication. Only prior art that falls 
under 102(e), (f), and (g) is subject to the 103(c) exclusion. Cox et al. is not prior art under 
102(e) because sub-section (e) only applies to patents and patent applications. Cox et al. is a 
research article and is a printed publication. Subsection (c) has nothing to do with published 
research articles. In addition, Cox et al. does not fall under 102(f) or (g) because these sub- 
sections deal with, in general, inventorship and interferences. Thus, applicant's arguments in 
regards to 103(c) are not applicable because the reference qualifies as prior art under 102(a). 
Applicant argues: 

Applicants respectfully submit that it is clear from the Cox et al. 
document itself that the subject matter in Cox et al., as „well as the 
claimed invention were, at the time of the claimed invention was 
made, owned by the same person or subject to an obligation of 
assignment under the same person. At the bottom of the first 
column of page 13 14 of the Cox et al. document it highlights that 
"the authors are with AT&T Labs-Research, Florham Park, NJ 
07932 USA." Each of the authors was well-known AT&T 
employees at the time of the invention, 
(middle of page 3 in filed response) 
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The examiner respectfully maintains that the rejections are proper because while the prior art 
reference of Cox was written by AT&T employees, MPEP section 715.01(b) states: 
The mere fact that the reference patent or application publication which shows but does 
not claim certain subject matter and the application which claims it are owned by the 
same assignee does not avoid the necessity of filing an affidavit or declaration under 37 
CFR 1 . 1 3 1 , in the absence of a show-ing under 37 CFR 1.132 that the patentee derived the 
subject matter relied on from the applicant (MPEP § 716.10). The common assignee does not 
obtain any rights in this regard by virtue of common ownership which he or she would not have 
in the absence of common ownership. In re Frilette, 412 F.2d 269, 162 USPQ 163 (CCPA 1969); 
Pierce v. Watson, 275 F.2d 890, 124 USPQ 356 (D.C. Cir. 1960); In re Beck, 155 F.2d 398, 69 
USPQ 520 (CCPA 1946). 

In addition, if the authors were well-known AT&T employees at the time of the invention and 
the prior art was owned by the same assignee, then it not readily apparent why applicant did not 
disclose this research article as required under rule 56. The subject matter in the Cox et al. is 
clearly relevant and related to the present invention, but was not disclosed on a form 1449 prior 
to it being cited by the examiner. It was available publicly at the filing date of applicant's 
invention. 

Furthermore, Appendix A filed by applicant may be necessary but is not sufficient to 
establish that applicant's conceived of the invention prior to the publication of Cox et al. 



Conclusion 
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1 . THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 
CFR 1 .136(a) will be calculated from the mailing date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS from the mailing 
date of this final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to DANIEL F. HAJNIK whose telephone number is (571)272-7642. 
The examiner can normally be reached on Mon-Fri (8:30A-5:00P). 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Ulka J. Chauhan can be reached on (571) 272-7782. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 



Application/Control Number: 10/662,550 Page 10 

Art Unit: 2628 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would 
like assistance from a USPTO Customer Service Representative or access to the automated 
information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

/Ulka Chauhan/ 

Supervisory Patent Examiner, Art Unit 2628 
DFH 



